With the advent of a mobile society, portable devices such as cellular telephones and laptop computers have become ubiquitous. Increasingly, such devices are used in a variety of teleconferencing situations in which the user is present at the office, at home, or while traveling. Cameras, which have fairly recently been incorporated in these devices, permit users to videoconference. In certain circumstances, it is desirable to replace the actual background of the user with a different background. However, the limited processing ability of the mobile devices often limits the ability of these devices to provide realistic background substitution. The realism of the substitute background is further decreased in situations in which the mobile device, and thus the integral camera, is not stationary. One problem lies in that as the camera moves, it is expected that the angle and position of the objects in the substitute background will change in a similar fashion. Motion due to the camera movement should thus affect all elements in the substitute background. To date, however, mobile devices do not have such capability; if the mobiles device moves, the foreground (user) will move without a similar motion of background. Moreover, in an environment that contains moving objects, such as a room with a window or an airport, moving elements in the original background that are alien to the substitute background will not be removed and thus will appear and disappear in the substitute background. It is desirable to overcome such problems and accordingly be able to provide a more realistic background substitution.